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Abstract 

The U.S. Department of Education Teacher 
Incentive Fund (TIF) seeks to transform education 
compensation systems so that principal and 
teacher performance (measured through classroom 
productivity measures) connects to compensation. 
Classroom-level productivity measures require robust 
student-teacher linkage data. Organizations such as 
the Value-Added Research Center, the Data Quality 
Campaign, and Battelle for Kids have recently been 
paying a great deal of attention to data quality. This 
paper presents findings from an exploratory study of 
eight TIF grantees and discusses how they acquired, 
verified, and managed their student-teacher linkage 
data through system development. Findings suggest 
that (a) grantees have multiple avenues for linkage 
data acquisition, and (b) all TIF grantees must 
devote considerable effort to verifying the validity 
of student-teacher linkage data. Furthermore, 
most of the TIF grantees included in this study are 
making progress in system development. This paper 
summarizes findings as a series of six lessons learned 
that may inform future and on-going compensation 
reform projects. 


Purpose and Objectives of the Study 

This paper reports on the ways in which Teacher 
Incentive Fund (TIF) grantees have collected, 
verified, and managed student-teacher linkage data 
to meet their TIF project needs. While this paper 
will benefit and inform current and future TIF 
grantees, it will also benefit districts and states that 
are committed to measuring student growth at the 
classroom level. In other words, this paper should 
be relevant to any district attempting to develop 
classroom-level measures of productivity (e.g., value- 
added) because these measures demand that analysts 
and leaders have access to high-quality student- 
teacher linkage data. This paper uses a framework for 
data quality that is informed by Watson, Kramer, & 
Thorn (2009) as well as organizations like the Data 
Quality Campaign (DQC, 2007; DQC 2010) and 
Battelle for Kids (2009). This framework focuses on 
the following areas of inquiry: 

A. Acquisition of student-teacher linkage data 

B. Verification of student- teacher linkage data 

C. Systems development to improve and sustain 
quality of student-teacher linkage data 

Through these areas, the authors examine the 
capacity of TIF grantees to collect student-teacher 
linkage data and maintain student-teacher linkage 
data quality. 

TIF, established in 2006, marked the first 
federal initiative to reform teacher and principal 
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compensation systems. TIF aims to increase the 
number of quality teachers in high-poverty schools 
and reward teachers and principals for improved 
student achievement (U.S. Department of 
Education, 2008). In 2007, the U.S. Department 
of Education awarded $99 million to 34 school 
systems around the nation to design and implement 
strategic compensation programs (U.S. Department 
of Education, 2008). These 34 TIF programs affect 
approximately 55,000 teachers and 2,500 principals 
in 18 states and include 109 school districts. 
Congress expanded funding for the program in 
FY 2010 with an additional appropriation of $437 
million, and the Department of Education finalized 
a third round of funding for 62 additional grantees 
(including some continued support for existing 
grantees) in late September 2010. 

All TIF projects have significant information 
technology (IT) requirements because grantees 
must be able to draw data needed to make merit- 
pay decisions from a variety of sources (e.g., human 
resources systems, student information systems, 
and teacher and principal observation records). 

Data quality problems that may be isolated in 
individual data systems can magnify as states and 
districts integrate data across systems and use them 
for high-stakes decisions. In particular, data about 
which teachers are teaching each content area to 
which students are particularly important to TIF 
projects. While school systems usually determine 
performance-based compensation using multiple 
measures, classroom-level measures of productivity 
account for 33% to 50% of the performance-based 
compensation determination under many of the 
plans currently in place. Consequently, without 
high-quality student-teacher data, TIF programs 
will struggle to make even the most basic progress 
toward compensation reform because they will 
be unable to connect the production of learning 
with the knowledge of educators in their system 
(Thorn, 2001). 


Methodology of the Study 

The TIF program includes 33 grantees from the 
original grantee cohorts. We purposefully selected 
8 of the 33 for this study based on their size, type 
of measurement, and award determination. (See 
http://cecr.ed.gov/initiatives/grantees/profiles.cfm 
for TIF grantees.) The grantees ranged from small 
local education agencies (LEAs) to large urban or 
state education agency (SEA) programs. The type of 
measurement varied from value added to attainment 
at the school, grade, and classroom levels. Grantees 
also varied in the amount and proportion of 
incentive allocated from test data, teacher evaluation, 
and other variables. 

The authors of this paper work for CECR and 
have worked with all TIF grantees as technical 
assistance providers. This project used qualitative 
methods that included document review {grant 
proposals, progress reports, memos, e-mails between 
CECR technical assistance providers and grantees) and 
interviews. We constructed interview protocols to 
probe the three areas of our data quality framework 
{acquisition, validation, and system development) . 

We conducted participant interviews over the 
phone with school and district staff who collect 
and manage data work for their respective TIF 
projects. We recorded, transcribed, and loaded 
the interviews into NVivo 8 for thematic analysis. 
When possible, we triangulated the reviewed 
documents with interviews. 

Phone interviews with each of the eight grantees 
(see Table 1 below) provided a rich set of 
qualitative data from diverse perspectives. Some 
participants were IT professionals, while others were 
administrators and practitioners. Additionally, some 
participants were district employees, while others 
were external consultants or contractors district(s) 
hired to develop data system capacity. (Note Table 
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1 uses pseudonyms for the grantees to protect the 
confidentiality of interviewees.) 

Interviews lasted approximately 40-45 minutes, 
and we recorded answers by both handwritten 
notes and digital recording device. Approximately a 
week before the interview, we provided respondents 


with a list of eight broad questions in three areas 
of inquiry. We conducted all interviews over the 
phone within an eight-day period in 2010. A single 
interviewer conducted approximately half of the 
interviews, while two or more colleagues conducted 
the remainder. 


Table I : Assessments, Measures, and Award Determination for Teacher Performance 




Type of Measurement 

Award Determination* 

Name 

# Schools 



Primary: Largest incentive amount 
Moderate: Moderate incentive amount 
Minor: Minor incentive amount 

VA 

Attain 

Anderson 

1 1 

S,G,C 

C 

T = Primary 
E = Minor 
A = Moderate 

Carlsberg 

20 

C 

S,C,G 

T = Primary 
A = Moderate 
O = Moderate 

Dipsonville 

230 

S.C 

S.G.G 

T = Primary 
H + O = Moderate 

Evergrace 

16 

s,c 


T = Primary 
E = Moderate 
A = Moderate 
H = Minor 

Hillside 

104 

s,c 

C 

T = Primary 
PD = minor 

Lakeview 

18 

S.C 

S 

T = Primary 

Northfork 

10 

S,G,C 

S.C 

T = Primary 
E = Minor 
A = Minor 
O = Minor 

Ogglethorp 

178 

s,c 


T = Primary 
E = Moderate 
A = Moderate 
PD = Moderate 

Sentinel 

46 

S.C 

s 

T = Moderate 
H = Moderate 
A = Minor 
PD = Minor 
O = Moderate 


* S = School-level, G = Grade-level, and C = Classroom-level measures. 

**T = % based on test data, E = % based on teacher evaluation data, H = % based on hard-to-staff schools or subjects, A = added 
responsibility, PD = participation in PD, O = other measures (i.e., National Board Certification, attendance rates, graduation rates). 
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Findings 

Common Themes 

Beyond gathering information on each of the three 
individual focus areas, we identified a number 
of challenges that were common to all areas and 
therefore qualify as overall themes. The first of these 
is the tension between complexity and simplicity, 
wherein grantees must balance the complex and 
varied characteristics of student-teacher relationships 
with the desire to keep their linkage process user- 
friendly, manageable, and timely. 

Another theme is the conflict between 
systemic approaches to data management and 
departmentalized approaches, wherein grantees 
must make tough decisions about the level of 
individualization that each system should allow. 
Allowing for detailed local differences in programs 
can radically increase complexity and challenge 
efforts to collect comparable data. 

A third tension involves the choice between point- 
in-time data and real-time data, wherein grantees 
must weigh an “ideal” linkage system against 
the intrusiveness of such a system on teachers, 
principals, and technology professionals. Readers 
should keep each of these tensions in mind when 
reading over the findings. 

We focus our findings in three areas: comparing 
methods of data acquisition, approaches for verifying 
data accuracy, and analyzing systems development. 


Data Acquisition 

All participants reported acquiring student-teacher 
linkage data from internal data systems, typically 
the district’s or states student information systems 
used to manage student schedules, attendance, 
grades, and discipline. Five of the participants 
reported a remarkably similar overall process, with 
an additional two grantees reporting only slight 
variations on this process. Only one grantee uses 
an entirely unique data acquisition system. 

There were several important differences in 
approaches. For example, one district reported using 
internal data systems, but only after hiring external 
partners to manage the process. Two participants 
reported using both internal data systems and 
external data systems (i.e., surveys asking teachers 
to self-report which students they taught). In these 
cases, districts reported using internal systems 
to populate electronic surveys which they then 
administered to teachers to verify student-teacher 
linkages. Once districts verified the data, they 
used this external system as the primary source of 
student-teacher linkage data. One grantee reported 
using an external data system to harvest student- 
teacher linkages directly, with no connection to 
internal systems. Districts developing processes 
for harvesting student-teacher linkage data should 
carefully consider if and how external partners might 
be able to improve how districts harvest student- 
teacher linkage data. 
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Data Verification 

Of primary importance in any “performance 
pay” system is establishing accurate links between 
students and teachers; only by properly attributing 
scores/ gains to the educators who helped create them 
can grantees ensure the integrity of compensation. 

It is no surprise, then, that one common theme 


among the respondents is the pains taken to 
validate their student-teacher linkage data. Every 
respondent described plans that involve at least one 
layer of verification and, in some cases, more. While 
these processes varied in their design — internal vs. 
external, individuals vs. committees, administrators 
vs. teachers — it is clear that every grantee feels that 
accuracy is a pressing matter. 


Table 2. Summary of Findings by EachTIF Grantee and Data Quality Area 


Name # Schools Links Source ^Leve^s* 0 " Complexity** Maturity*** 

Anderson 

1 1 

Internal 

3 

High 

Low 

Carlsberg 

20 

Internal 

3 

Medium 

Medium 

Dipsonville 

230 

External 

3 

Medium 

High 

Evergrace 

16 

Internal + 
consultant 

3 

Medium 

Medium 

Hillside 

104 

Both 

4 

High 

High 

Northfork 

10 

Internal 

4+ 

High 

Low 

Ogglethorp 

178 

Both 

3 

Medium 

Medium 

Sentinel 

46 

Internal 

1 (Pilot only) 

High 

Low 


*The number of distinct steps/persons grantees use to validate student-teacher links. 

** Grantee’s system’s handling of “atypical” education contexts (e.g.,team teaching, student mobility). 

*** Level of development of grantee’s data system, as a function of time/experience/piloting/live rollout/etc. 


While most respondents shared the same general 
approach to the first issue (seeking a balance 
between simplicity and complexity), they differed 
greatly in regard to the second issue — how they 
treat “atypical” settings such as team teaching or 
multidisciplinary courses. Some grantees establish 
arbitrary boundaries to simplify the process; for 
example, any teacher employed at least 50% FTE 
is eligible for the same award amounts as a full-time 
teacher. Some grantees spend resources creating 
intricate and multi-faceted attribution applications 
that assign each student individually (for example) 
to teachers in 10% increments. Some grantees rely 
on the professionalism of their educators, such as the 
use of teacher committees to collectively document 


more complex delivery of instruction. The rationale 
for using committees to document instruction 
delivery models is based on the assumption that 
those committees comprise the same members as the 
instructional delivery teams, making it more likely 
that the “team” will accurately attribute instruction 
to each member in the team. Finally, there is a 
group of districts that relies on principals to ascribe 
responsibility to teachers working in more complex 
settings. In any case, this particular area of student- 
teacher assignment bears watching, given that the 
“traditional” one teacher/one classroom/one year 
model may not be the dominant model of delivery 
in many locations. 
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Systems Development 

Grantees reported a number of ways in which their 
TIF project had become a driver for how the grantee 
strategically leverages his/her information technology 
infrastructure. In many cases, this focus on systems 
development extends to external partners like data 
system vendors and non-profit consultants as well. 
Grantees reported several ways in which their TIF 
project had changed how they interacted with 
internal technical staff. First, TIF grantees needed to 
understand the capacities and limits of their existing 
data systems and that staff in charge of overseeing 
the implementation and maintenance of those 
systems held that knowledge. Second, TIF grantees 
reported an overall increase in awareness and focus 
on the role of data within the grantee’s organization. 
As the perceived importance of student-teacher 
linkage data increased, so did the desire to ensure 
that the data were high quality. Third, TIF projects 
challenged grantees to connect data across previously 
unlinked data systems and departmental boundaries. 


TIF grantees also reported benefits and challenges 
associated with working with vendors and external 
partners. On the one hand, these partnerships 
added significant capacity to grantees in the 
area of managing and verifying student-teacher 
linkage data. On the other hand, partnerships 
sometimes were associated with increased project 
management complexity. Interviews revealed that 
sometimes inflexible contract deliverables and poor 
communication between both vendor and district 
strained relationships between vendors and schools. 
However, grantees generally reported more benefits 
from such partnerships than challenges. Most 
important, a common element among responses 
was the idea that the TIF grant mobilized internal 
improvement and external investment. In this sense, 
complying with the grant’s technology requirements 
drove innovation and increased grantees’ student- 
teacher links capacity — thereby strengthening the 
vital and initial link in the performance-based 
compensation chain. 
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Lessons Learned 

The findings of this project provide valuable lessons 
to other districts and states seeking to implement 
compensation reform projects. Some of these lessons 
focus on organizational and leadership issues. Some 
are technical demands that arise as TIF projects face 
the challenge of coordinating how to use technology 
to implement reformed compensation systems. 
Other lessons focus on important process and 
implementation variables. Many are combinations 
of all three. 

Lesson #1: 

Plan for systems integration 
with stakeholder input. 

Compensation reform requires TIF grantees to 
integrate student information system data with 
human resources and payroll system data within 
a delivery system that supports verification. 

Grantees typically obtain student-teacher linkage 
data from the student information system, but 
need to integrate these data with data from human 
resources and payroll systems. However, grantees 
did not initially have these systems designed 
with performance-based compensation in mind. 
Therefore, grantees need to take care to ensure that 
they do not use the data in ways that are unfounded. 
For example, some districts may not maintain 
accurate employee data in their student information 
system, which would make it difficult to match 
employee records from the Human Resource system 
with student-teacher linkage data from the student 
information system. Including various stakeholders 
like technical leads, mid-managers, and personnel 
who know how schools use technology on a day-to- 
day basis will help roadmap potential problems. 


Lesson #2: 

Enhance trust and fidelity of student-teacher 
linkage data with good verification processes. 

It is important to realize the purpose and benefits 
of verification to a TIF project. First, the purpose of 
verifying student rosters and teacher assignments is 
to ensure that districts base pay-outs upon data that 
are as accurate as possible. Verification of student- 
teacher linkage data allows leaders to manage risks 
associated with incorrect payouts. Second, the 
process of verification provides an excellent window 
to view classroom practices that inextricably connect 
to measures of student growth. Data systems often 
do not record team teaching strategies such as shared 
classrooms, pull-out services, and push-in services. 
Verifying student-teacher relationships provides 
TIF grantees with a way of knowing about these 
approaches so that recipients perceive awards as fair. 

Lesson #3: 

Maintain balance between nuance and 
pragmatism in data collection and storage. 

Leaders must balance nuances such as measuring 
team teaching with pragmatism in order to optimize 
stakeholder buy-in and support while managing 
the complexity of compensation reform. Leaning 
too far too fast toward a nuanced data model for 
student-teacher linkages will increase the validity 
of the data used to determine pay-outs, but at 
the expense of significantly increasing project 
management costs. For example, measuring team 
teaching over time requires district leaders to query 
teachers at multiple times; furthermore, analyzing 
and using these data becomes inherently more 
complex as team teaching assessments increased 
in frequency. On the other hand, ignoring real 
world nuances such as team teaching for the sake 
of simplicity will likely undermine teacher buy-in 
and support because stakeholders will know that 
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the process of merit determination excluded team 
teaching strategies like pull-out and push-in teaching 
supports. Leaders should consider ways to gradually 
increase the data collection methods’ sensitivity 
to nuance over time so that they can spread out 
capacity-building and make it more manageable. 

Lesson #4: 

Leverage school personnel 
for different purposes. 

When planning for verification and data collection 
at the school and classroom levels, TIF leaders 
should consider what roles each type of school 
personnel will play. Principals are often an 
initial point of contact for verifying basic staff 
assignment and course offerings. Some TIF 
grantees also use principals to resolve remaining 
questions about who taught whom after initial 
assignment by teachers. On the other hand, 
grantees should also ask teachers to verify their 
student rosters and inform the district about team 
teaching. Districts may best manage tasks related 
to determining which courses qualify for math 
or reading content areas. 

Lesson #5: 

Connect TIF projects to IT infrastructure. 

Since TIF projects use data from systems not 
originally designed to inform compensation 
decisions, it is critical that TIF leaders work closely 
with information technology specialists and leaders 
throughout the life cycle of TIF projects. Technical 
staff may know where many data limitations may 
lie, as well as ways to mitigate those limitations. 
Technical staff may be aware of specific ways in 
which schools use their student information system 
on a day-to-day basis. This knowledge can help 
district leaders understand how best to integrate 


new tasks into how schools actually use student 
information systems. 

In addition, technical staff should facilitate 
the integration of data systems (e.g., merging 
student-teacher linkage data with human resource 
data). TIF projects provide district technical 
services leaders with an excellent opportunity 
to build technical capacity beyond compliance 
reporting and toward knowledge management 
and decision support. 

Lesson #6: 

Use external partners to increase 
both capacity and overhead. 

Many TIF projects interviewed for this study 
worked closely with external partners such as data 
system vendors, analytic service providers, and 
communication specialists. These relationships 
greatly enhanced district capacity to perform specific 
tasks such as verifying student-teacher linkage 
data, reporting award decisions, implementing 
new data systems, and of course calculating 
performance metrics. 

However, these relationships require management 
and increase the overhead costs of TIF projects. 

In some cases, when district staff had difficulty 
identifying clear roles, deliverables, and 
communication norms, the costs of managing 
external partners have outweighed the benefits 
of the relationship. TIF leaders should think 
strategically about external partner relationships 
and plan for those relationships by defining 
deliverables, timelines, and communication clearly. 

Within the domain of performance-based 
compensation, current and future TIF leaders 
need to understand how their data systems collect 
and manage student-teacher linkage data and how 
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to integrate technical experts into the design and 
implementation phases of their projects. 

Additionally, TIF leaders need to speak with data 
systems experts about what might compromise 
the quality of student-teacher linkage data. 

Watson, Kraemer, and Thorn (2009) provide a rich, 
real-world description of how some TIF grantees 
developed and implemented their systems to harvest 
student-teacher linkages and outline the challenges 
of connecting student to teachers in increasingly 
complicated educational contexts. 

Conclusions 

One major finding of this study was that TIF 
grantees have needed to engage in capacity-building 
activities in order to be able to access and manage 
student-teacher linkage data of sufficient quality. 
Historically, districts have not designed their data 
systems to inform high-stakes teacher decisions 
like performance-based compensation. Instead, 
districts designed their K12 data systems for funding 
allocation decisions and compliance reporting. 

Prior to implementation, most K12 districts simply 
have not had any reason to systematically collect 
or validate student-teacher linkage data. New TIF 
grantees should plan accordingly to address how 
they will acquire, manage, and validate student 
teacher linkage data. 

Findings also suggest that district data systems are 
likely capable of establishing initial best guesses 
about who is teaching whom, although districts 
may need help in extracting and managing student- 
teacher linkage data. TIF project leaders should plan 
to include technical experts, school programming 


specialists, and research and reporting staff early 
in their project design and implementation to 
best understand their district’s capacity to harvest 
student-teacher linkage data from district data 
systems. Part of this process will result in decisions 
about what factors to include or exclude in their 
TIF design and implementation (e.g., team teaching 
variables, mobile students). 

Findings of this study suggest that districts will 
likely benefit from external partnerships that help 
districts acquire, manage, and verify student-teacher 
linkage data. However, districts must manage these 
types of partnerships with care, thoughtfulness, and 
good communication. 

Finally, it is worth noting that the utility of 
student-teacher linkage data should not be limited 
to districts implementing performance-based 
compensation systems. Organizations such as 
the Data Quality Campaign 1 and the Schools 
Interoperability Framework Association 2 are 
increasingly focusing on student-teacher linkages 
as a necessary condition for using data to drive 
quality and improvement. A burgeoning body 
of literature is espousing the benefits of accurate 
student-teacher linkages — benefits such as improved 
reporting capacity, identification of effective programs 
and practices, improved feedback systems (DQC, 
2007), and improving teacher quality (DQC, 2010). 
Likewise, Battelle for Kids has also long recognized 
the importance of accurately linking teachers to 
students (Battelle for Kids, 2009). While student- 
teacher linkage data are not the only key factor in 
understanding what works in education, these data 
are necessary for attributing outcomes in student 
achievement to interventions at the teacher level. 

1 www.dataqualitycampaign.org 

2 The Schools Interoperability Framework Association (SIFA) is a data- 
sharing specification originally developed to allow information systems 
within K-12 districts to exchange data without requiring wholesale 
replacement of existing systems. See http://www.sifa.org 
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